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ABSTRACT 

Many states and school districts are making concerted 
efforts to boost student achievement in mathematics and science. This 
information brief is intended for those who seek to improve student 
learning by creating coherent systems of expectations and assessments 
in states and districts. Other potential audiences are those who 
study reform, make decisions about reform, and are affected by 
reform. The intention of this brief is to help people think more 
clearly about the concept of alignment and to help them examine what 
is required for expectations and assessments to be in alignment. The 
major elements of an education system must work together to help 
students achieve higher levels of mathematical and scientific 
understanding. Topics discussed include the importance of alignment, 
methods of alignment including sequential development, expert review, 
and document analysis; and specific criteria including content focus, 
articulation across grades and ages, equity and fairness, pedagogical 
implications, and system applicability. It is concluded that the 
alignment of expectations and assessments is a key underlying 
principle of systemic and s tandards - bas ed reform. Establishing 
alignment among policy elements is an essential activity for 
improving the potential for realizing significant reform. (JRH) 
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REPORTING ON ISSUES AND RESEARCH IN SCIENCE, MATHEMATICS, ENGINEERING, AND TECHNOLOGY EDUCATION 



Determining Alignment of Expectations and Assessments 
in Mathematics and Science Education 1 

By Norman L. Webb 



M any states and school districts are making concerted 
efforts to boost student achievement in mathematics 
and science. These are not simple face lifts, but 
attempts to develop deep, lasting changes in how students learn 
these critical subjects. 



Vertical and Horizontal Alignment 
within an Education System 



Policy 




I 

Practice 



I 

Student Outcomes 



This Brief is intended for those who seek to improve student 
learning by creating coherent systems of expectations and assess- 
ments in states and districts. Other potential audiences are those 
who study reform, make decisions about reform, and are affected by 
reform. The intention of this Brief is to help people think more 
clearly about the concept of alignment, and to help them examine 
what is required for expectations and assessments to be in alignment. 



Why Alignment Is Important 

Educators, notably through efforts spearheaded by national profes- 
sional associations, increasingly recognize the need for major reform 
in K-12 mathematics and science curricula, and are embracing a 
vision of ambitious content for all students. Making this vision a 
reality means encouraging “a far deeper and more dynamic level of 
instructional decision making” (Baker, Freeman, & Clayton, 1991), 
something that cannot be done simply by mandating new account- 
ability measures. At the heart of these efforts to make deep changes 
in instruction is the concept of “alignment.” The major elements of 
an education system must work together to help students achieve 
higher levels of mathematical and scientific understanding. 

Educators increasingly recognize that, if policy elements are not 
aligned, the system will be fragmented, send mixed messages, and 
be less effective (CPRE, 1991; Newmann, 1993)." For example, the 
Systemic Initiatives program of the National Science Foundation 
seeks to help states, districts, and regions establish policies based, in 
part, on assessments aligned with those goals. Other examples: The 
U.S. Department of Education’s explanation of Goals 2000: 
Educate America Act, and the Improving America’s Schools Act 
(which includes Title 1), both say that alignment of curriculum, 
instruction, professional development, and assessments are key 
performance indicators for states, districts, and schools that are 
striving to meet challenging standards. 

As more and more attention is paid to the accountability of edu- 
cation systems, alignment between assessments and expectations for 
learning becomes not only critical, but also essential. Just as a 
schooner’s speed increases when its sails are set properly, alignment 
among an education system’s policy elements will strengthen that 
system, and improve what the system is able to attain. Alignment is 
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y 4' ^ critical to helping an education system articu- 
^ late and maintain its desired course and 
( yjj^ intensity. An aligned system is better able to 
p' w ^ focus [ls ^sources and thereby strengthen its 
' capacity for making deep, meaningful changes in 
instructional decision making and practice. Alignment also serves to 
keep local policy efforts in synch with larger-scale initiatives. 

This Brief focuses on alignment between two major elements of 
education policy: 

• Expectations of what students should know about mathematics 
and science and what they should be capable of doing with that 
knowledge. Expectations can be communicated in different ways. 
Educators can, for example; craft sets of standards or frameworks, 
ranging from broad vision statements to precise indications of 
expected performance and recommended instructional practices. 

• Assessments that accurately gauge student achievement in science 

and mathematics and indicate whether expectations are being 
achieved. Assessments can be used to formulate policy, monitor 
policy effects, enforce compliance with policies, demonstrate 
accountability, make comparisons, monitor progress toward 
goals, and/or make judgments about the effectiveness of partic- 
ular programs (Blank, Pechman, & Goldstein, 1996). 

There are, of course, many other important elements in any educa- 
tion system, including professional development, instructional 
materials, college entrance requirements, teacher certification, 
resource allocations, and state mandates. But this Brief focuses on 
expectations and assessments because those elements are now of 
great concern among educators and policymakers, and because 
those are the elements at the center of most thinking about align- 
ment to date. 



Methods of Alignment 

Determining alignment between expectations and assessments is 
difficult for several reasons. To begin wirh, both expectations and 
assessments frequently are expressed in several pieces or docu- 
ments, making it difficult to assemble a complete picture. Also, it 
is difficult to establish a common language for describing different 
elements of policy. The same term may have very different mean- 
ings when used to define a goal and when used to describe some- 
thing measured by assessment. Further, the policy environment in 
an education system can be constantly changing. New goals can be 
mandated, for example, while old forms of assessment are still in 
place. Ever-expanding content areas, expanding technology, and a 
growing body of research on learning also can contribute to the 
complexity of identifying expectations and assessments. 

A review of current practice and relevant literature identifies 
three major approaches to assuring alignment. These are not the 



only approaches, however, nor should they be seen as items on a 
menu to be chosen and then applied in pure form. In most situa- 
tions, some combination of these approaches is appropriate. 

Sequential Development. Policy elements, such as expectations 
and assessments, are aligned by design. A set of standards, for 
instance, might be converted directly into specifications for devel- 
oping an assessment. Once one policy element is established, it 
becomes the blueprint for subsequent elements. For example, the 
South Carolina Department of Education (1996) approved stan- 
dards in a content area that are used to develop academic achieve- 
ment standards (measurable outcomes), which are then used to 
develop assessment instruments. 

One disadvantage to this approach is the amount of time needed 
to put a sequentially developed program in place. This approach 
also ignores a synergism among policy elements: The development 
of assessments, for example, can provide useful information for 
thinking about instruction and what students can be expected to 
learn. Another disadvantage to this approach is that it frequently 
does not reflect reality: In many states, the process for developing 
expectations and assessments is not linear or sequential, but 
more dynamic. 

Expert Review. A panel of experts reviews the policy elements 
and makes some judgment on their alignment. For example, the 
Oregon Department of Education convened a national panel to 
look at various issues related to its standards (Roeber, 1996). A 
subpanel looked at the alignment of the planned assessments and 
the standards. 

The format and formality of this process can vary. In many 
states the process is an open one, seeking input from committees 
and community forums of teachers, administrators, parents, and 
others. Whatever format is used, however, must include input 
from content-area specialists, because the comparisons to be made 
are complex and require sophisticated knowledge about how 
students learn. 

Document Analysis. Alignment can be measured by coding and 
analyzing the documents that convey the expectations and assess- 
ments. A coding system must be developed that specifies the distinc- 
tions to be made in describing each document. The documents are 
then divided into blocks, such as individual standards, which can be 
described separately using the coding system's categories. Coders must 
be trained to independently and validly describe the documents by 
using the coding categories on the blocks of document information 
(Schmidt & McKnight, 1995; Porter, 1995.) For example, the Third 
International Mathematics and Science Study successfully trained 
national teams to perform document analyses comparing curriculum 
materials with assessments used in the study (McKnight, Britton, 
Valverde, & Schmidt, 1992). 
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These approaches and their interactions raise 
questions about quality control. Sequential develop- 
ment, for example, frequendy is controlled within 
an agency and therefore is less likely to include any 
external review. While such reviews add authority, 
they can’t always be done within the short time lines 
required by legislative mandates or administrative 
pressures. The quality of expert review, on the other 
hand, depends on how qualified the reviewers are, 
and whether they have the opportunity to interact 
and build consensus. And the quality of document 
analysis depends on the validity of the scoring rubric 
being used, the quality of training, and the reliability 
of the coders. 

Most likely, these approaches will be used in con- 
junction with each other. One approach will be used 
to verify another, or two or three approaches will be 
used together. An expert panel, for example, may use 
document analysis to judge alignment. 

Specific Criteria 

These approaches to the judging of alignment are 
strengthened by using specific criteria to assure 
agreement among expectations and assessments. 
The following criteria were identified through a 
review of national and state standards and align- 



ment studies. They were adjusted after review by a 
panel of assessment experts from the National 
Institute for Science Education and the Council of 
Chief State School Officers, state curriculum super- 
visors, and others. It is expected that this set of 
criteria will evolve as they are used. 

The five categories are intended to be a compre- 
hensive set for judging the alignment between 
expectations and assessments. Each general category 
and all subcategories are important in ascertaining 
the coherence of a system, meaning the degree to 
which assessments and expectations converge to 
direct and measure student learning. In practice, 
reaching full agreement between expectations and 
assessments on all criteria is extremely difficult. 
Tradeoffs must be made because real constraints 
exist on any education system, including resources, 
finances, time, and legal authority. Decision makers 
must consider potential consequences when decid- 
ing what tradeoffs to make among these criteria, or 
what level of compliance will be acceptable. 

Such decisions will hinge on a number of factors. 
Assessing the depth of content knowledge, for 
example, can conflict with assessing the breadth of 
knowledge (these concepts are explained in greater 
detail below). Given finite resources, it may be 




The major elements of 
an education system must 
work together to guide 
the process of helping all 
students achieve higher 
levels of mathematical and 
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Assessments must 
achieve a high 
degree of match 
between what 
students are 
expected to know 
and what 
information is 
gathered on 
their knowledge. 



difficult to fully explore both. Decision makers will 
need to choose which criteria are considered uiore 
important within a particular context and why, and 
how those decisions affect the pursuit of alignment. 

Because resources are finite, decision makers also 
will need to think broadly about expectations and 
assessments. It may be far more reasonable and cost- 
efficient, for example, to give teachers the responsi- 
bility of assessing students’ abilities at reasoning and 
problem-solving, instead of trying to measure them 
through new systemwide tests. Whether assessments 
are carried out at the classroom level, locally, or 
systemwide, however, the focus must be the same: 
achieving a high degree of match between what 
students are expected to know and what informa- 
tion is gathered on their knowledge. 

The following criteria are ordered to consider 
content first, then students, instruction, and finally 
system concerns. 

1. CONTENT Focus. Expectations and assess- 
ments should focus consistently on developing 
students’ knowledge of mathematics and science. This 
consistency will be present to the extent expectations 
and assessments share the following attributes: 

A. Categorical Concurrence. The same cate- 
gories of content, such as subject headings 
and their subheadings, appear in each. The 
level of detail, however, may vary: Standards 
and frameworks can be statements of general 
expectations, or they can be more refined 
descriptions of content. Assessment docu- 
ments may be still more specific. To be in 
categorical concurrence with the National 
Science Education Standards developed by the 
National Research Council (1996), for exam- 
ple, an assessment would need to represent 
each of the eight content topics in those stan- 
dards. Alignment would be even greater if 
the assessment results were reported by those 
eight content topics. 

B. Depth of Knowledge Consistency. This can 
vary on a number of dimensions, including 
the level of cognitive complexity of the infor- 
mation students should be expected to know, 
how well they should be able to transfer this 
knowledge to different contexts, and how 
much prerequisite knowledge they must have 



in order to grasp more sophisticated ideas. 
Expectations and assessments are aligned if 
they reflect similar requirements on these 
dimensions. For example, the Curriculum 
and Evaluation Standards for School 
Mathematics published by the National 
Council of Teachers of Mathematics (1989) 
states that students in grades 9 through 12 
should study data analysis and statistics, so 
that all students can “design a statistical 
experiment to study a problem, conduct the 
experiment, and interpret and communicate 
the outcomes.” An assessment system requir- 
ing students only to interpret an existing set 
of data would not be aligned with the depth 
of knowledge specified in this standard. 

C. Range of Knowledge Correspondence. Expec- 
tations and assessments cover a comparable 
span of topics and ideas within categories. For 
example, standards published by the Virginia 
Board of Education (1995) say students 
should be able to read four different types of 
maps: bathymetric, geologic, topographic and 
weather. For an assessment to correspond to 
that range of knowledge, it would need to 
measure how well students can interpret infor- 
mation using all four map types. 

D. Structure of Knowledge Comparability. The 
underlying concepts of science and mathe- 
matics, and what it means to “know” these 
concepts, are in agreement. For example, if 
standards indicate that students should see 
mathematics “as an integrated whole” 
(NCTM, 1989) or “science as inquiry” (NRC, 
1996), then the assessment activities should 
be directed toward those same ends. Both 
expectations and assessments should embody 
similar requirements for how students are to 
draw connections among ideas. Assessment 
of knowledge only as isolated skills, for 
example, would not be in full alignment with 
the national standards. 

E. Balance of Representation. Similar emphasis 
is given to different content topics, instruc- 
tional activities, and tasks. The expectations 
and assessments give comparable emphasis to 
what students are expected to know, what 
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they should be able to do, and in what con- 
texts they are expected to demonstrate their 
proficiency. For example, the National Science 
' Education Standards emphasizes different 
skills at different grade levels: Students in 
kindergarten through grade 4 should focus on 
developing observation and description skills, 
while students in higher grades should work 
on constructing models that explain visual 
and physical relationships. An aligned assess- 
ment system would need to reflect a similar 
^ shift in emphasis. It also would need to 
include enough different tasks or activities to 
reflect the same priorities and intentions. 

F. Dispositional Consonance. When expecta- 
tions include more than learning concepts, 
procedures, and their applications — such 
as molding student attitudes and beliefs 
about science and mathematics — assess- 
ments also should support that broader 
vision. For example, the National Science 
Education Standards underscores the impor- 
tance of students becoming self-directed 
learners. The ability to self-assess under- 
standing is an essential tool for this process. 
Assessment practices aligned with this goal 
will include opportunities for students to 
critique their own work and to explain how 
' work samples provide evidence of under- 
standing. Teachers need to give students 
opportunities to reflect on their scientific 
understanding and abilities, so they can 
begin to internalize the expectation that 
they can learn science. 

2. Articulation Across Grades and Ages. 
Students’ knowledge of mathematics and science 
grows over time. Expectations and assessments 
should be rooted in a common view of how 
students develop and how best to help them learn at 
different developmental stages. This common view 
should be based on: 

A. Cognitive Soundness Determined by Best 
Research and Understanding. There has 
been considerable research on the learning of 
mathematics and science, which has pro- 
duced extensive knowledge of how students 
mature in their understanding of these 



Expectations and assessments that are aligned will demand equally 
high learning standards for all students, while providing fair means 
for all students to demonstrate the expected level of learning. 

content areas (Romberg & Carpenter, 1986; 

Stein, Grover, & Henningsen, 1996). 

Expectations and assessments should build 
on this knowledge to develop a sound learn- 
ing program, and they should do so in ways 
that are aligned. 

B. Cumulative Growth in Knowledge During 
Students’ Schooling. Expectations and assess- 
ments should be linked by an underlying 
rationale of mathematics and science as 
content areas. Although the learning of math- 
ematical and scientific concepts over time 
doesn’t follow a strict order of steps, students 
often need to grasp certain concepts and ideas 
in order to address more advanced ideas. For 
students to take part in scientific inquiry, for 
example, they first need to learn to identify 
questions and concepts that guide scientific 
investigations, how to design and conduct 
such investigations, how to use technology 
and mathematics, how to formulate and revise 
scientific explanations, and how to recognize 
and evaluate alternative explanations. Aligned 
expectations and assessments describe and 
represent, in complementary fashion, the 
underlying structure of knowledge students 
need to develop and how their instructional 
experiences should be organized. 

3. Equity and Fairness. When expectations are 
that all students can learn to high standards, aligned 
assessments must give every student a reasonable 
opportunity to demonstrate attainment of what is 
expected. Expectations and assessments that are 
aligned will demand equally high learning standards 
for all students, while providing fair means for all 
students to demonstrate the expected level of learn- 
ing. The knowledge a student will demonstrate on 
an assessment can vary by the form of assessment 
(Baxter, Shavelson, Herman, Brown, & Valadez, 

1993). Even a slight variation in the wording of a 
question can alter performance. Rarely will one form 
of assessment be capable of producing valid evidence 
for all students. 
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A students ability to perform well on an assess- 
ment depends on a number of factors in addition to 
the level of knowledge, including culture, 
J social background, and experiences. 
Therefore, expectations and assessments 
will be better aligned, and more equitable, 
if multiple forms of assessment are used. The challenge 
becomes developing and maintaining an aligned system with a 
variety of means of assessment, which function together to reflea 
more accurately what students know and can do. 

It may be difficult to gauge the alignment of expectations and 
assessments on these criteria for some time. Consistently low scores 
on an assessment of a particular learning goal may be the result of 
.„many factors, including misplaced expectations, rather than poor 
instruction or lack of effort by students. Students may be develop- 
mentally unprepared to attain a particular expectation, for example, 
or the structure of the curriculum may keep them from attaining 
sufficient experiences to learn what is expected. It takes time for 
patterns to form and be recognized. 

4. Pedagogical Implications. Classroom practice greatly influ- 
ences what students learn. Expectations and assessments can and 
should have a strong impact on these practices and should send 
consistent messages to teachers about appropriate pedagogy. 

Judging the pedagogical implications of expectations and assess- 
ments requires more than simple content analysis. Any review must 
attempt to gauge the likely implications for classroom practice. 
Meaningful analyses have been done, for example, by directly ask- 
ing teachers how they interpret expectations and assessments and 
how their classroom practices fit with them (Romberg, Zarinnia, & 
Williams, 1990; Cohen, 1990). 

Of course, the true test is what happens in the classroom. For 
example, educators are now paying increased attention to the 
importance of involving students in scientific inquiry, hands-on 
learning, and more “authentic” instruction (Newmann, Secada, & 
Wehlage, 1995). Assessments that reflect a more passive type of 
instruction would be less aligned with those expectations. Likewise, 
expectations that students should perform scientific inquiry 
through actively constructing ideas and explanations (NRC, 1996) 
will lack full alignment with assessments that are based solely on an 
assumption that students have memorized canonical ideas and 
explanations. Alignment is achieved when the instructional prac- 
tices and materials implied by expectations, and those implied by 
assessments, are consistent. 

Critical elements to be considered in judging alignment and its 
influence on pedagogy include: 

A. Engagement of Students and Effective Classroom Practices. 
Traditional forms of student assessment, and the constraints 
imposed by limits on time and other resources, may place an 
inordinate influence on the superficial acquisition of skills 




and facts. In this way, education systems can gravitate toward 
readily measured outcomes, instead of more complex but 
also more desirable outcomes, such as students being able to 
investigate, create models, or otherwise demonstrate deeper 
content knowledge. 

Expectations and assessments need to work together to 
provide consistent messages to teachers, administrators, and 
others about the goals of learning activities. For example, a 
preliminary draft of statewide academic standards for Illinois 
indicates that students should learn and contribute produc- 
tively both as individuals and as members of groups (Illinois 
Academic Standards Project, 1996). This is defined in the 
draft as an important skill, one that will greatly determine 
the success of students later in life. But if no part of the 
assessment system produces evidence of whether students are 
contributing productively as members of groups, then 
teachers would receive conflicting messages about how 
much classroom time should be spent having students 
work in teams. 

B. Use of Technology, Materials, and Tools is vital to knowing 
and “doing” mathematics and science today. Students should 
develop skill and confidence using tools such as calculators 
and computers in their everyday lives (National Council of 
Teachers of Mathematics, 1991). Expectations and assess- 
ments should send students consistent messages about 
technology and how it is related to what they are expected to 
learn. If standards indicate that students should learn to use 
calculators or computers routinely, for example, then the cur- 
riculum should provide adequate opportunity for students 
to use them in this manner. To be aligned, assessments 
would allow students to use calculators and computers 
effectively to derive correct answers. 

5. SYSTEM Applicability. Although expectations and assessments 
should seek to encourage high student performance, they also need 
to form the basis for a program that is realistic and manageable in the 
real world. The policy elements must be in a form that can be used 
by teachers and administrators in a day-to-day setting. Also, the 
public must feel that these elements are credible, and that they are 
aimed at getting students to learn things about mathematics and 
science that are important and useful in society. 

Conclusions 

Above all else, when using these criteria to judge the alignment of 
expectations and assessments in a system, a sense of reality needs to 
be maintained. The available resources, the amount of time avail- 
able, legislative mandates, and other factors will influence how 
well alignment can be determined and how practical it is to make 
such determinations. 

The alignment of expectations and assessments is a key underly- 
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ing principle of systemic and 
standards-based reform. Establishing align- 
ment among policy elements is an early 
activity for improving the potential for real- 
izing significant reform. Those working to 
build aligned systems should not think too 
narrowly about the task. The criteria 
presented here demonstrate that a number 
of factors can be considered in judging 
alignment among policy elements. These 
can be studied in several alternative and 
potentially complementary ways. 

In approaching reform, the considera- 
tion of alignment cannot come too soon. 
And just as educators need to remain 
vigilant to assure that expectations, assess- 
ments, and instructional practices are 
current, they also will need to review the 
alignment among these major policy 
elements as new policies are instituted, new 
administrative rules are imposed, and 
system needs are changed. 

❖❖❖ 
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is the degree to which the elements of an education system are 
aligned with other forces, such as national standards, public 
opinion, work force needs, textbook content, classroom instruc- 
tion, and student outcomes. 

^ Alignment is intimately related to the “validity” of tests, but 
distinctions can be drawn between the two concepts. Alignment 
refers to how well all policy elements in a system work together 
to guide instruction and, ultimately, student learning. Validity, 
on the other hand, refers to the appropriateness of inferences 
made from information produced by an assessment. For exam- 
ple, the degree to which a test is aligned with a curriculum 
framework may affect the test’s validity for a single purpose, such 
as-making decisions on the curriculum’s effectiveness. But a test 
and a curriculum framework that are in alignment will work 
together to communicate a common understanding of what 
students are to learn to provide consistent implications for 
instruction, and to represent fairness for all students, and will 
be based on sound principles of cognitive development. 

^ A more complete discussion of these criteria, and how they can 
be used, is available in Webb, N. L., Criteria for Alignment of 
Frameworks , Standards and Student Assessments for Mathematics 
and Science Education. This paper is a joint publication by the 
National Institute for Science Education and the Council of 
Chief State School Officers. For more information, contact NISE 
at (608)263-1028 or via the NISE World Wide Web site: 
http://www.wcer.wisc.edu/nise. 

5 The complete paper includes a more detailed description of 
procedures and scales useful for judging attainment of these 
criteria. 
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